User/object interactions in an augmented reality environment

ABSTRACT

An augmented reality environment allows interaction between virtual and real objects. By monitoring user actions with the augmented reality environment various functions are provided to users. Users may buy or sell items with a gesture, check inventory of objects in the augmented reality environment, view advertisements, and so forth.

BACKGROUND

Augmented reality environments allow interaction among users andreal-world objects and virtual or computer-generated objects andinformation. This merger between the real and virtual worlds paves theway for new interaction opportunities.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Theuse of the same reference numbers in different figures indicates similaror identical components or features.

FIG. 1 shows an illustrative scene within an augmented realityenvironment which includes with an augmented reality functional node andassociated computing device.

FIG. 2 shows an illustrative augmented reality functional node thatincludes a computing device, along with other selected components.

FIG. 3 illustrates an environment encompassing the computing device,users, advertisement servers, merchants, and so forth.

FIG. 4 is an illustrative diagram of an object datastore configured tostore information about objects in the augmented reality environment.

FIG. 5 is an illustrative diagram of a user interaction datastoreconfigured to store information about how users interact with objects inthe augmented reality environment.

FIG. 6 is an illustrative diagram of a transaction datastore configuredto store information about transactions involving objects in theaugmented reality environment.

FIG. 7 is an illustrative diagram of an advertisement datastoreconfigured to store information about advertisements associated withobjects.

FIG. 8 is a schematic of a user interface module.

FIG. 9 is a schematic of a transaction module.

FIG. 10 is an illustrative augmented reality environment.

FIG. 11 is an illustrative process of tracking user interactions in theenvironment.

FIG. 12 is an illustrative process of inventory management in anaugmented reality environment.

FIG. 13 is an illustrative process of assisting an activity in anaugmented reality environment.

FIG. 14 is an illustrative process of receiving objects into anaugmented reality environment and replenishing those objects.

FIG. 15 is an illustrative process of initiating a purchase in anaugmented reality environment.

FIG. 16 is an illustrative process of initiating an offer for transferof property rights in an augmented reality environment.

FIG. 17 is an illustrative process of presenting advertisements in anaugmented reality environment.

DETAILED DESCRIPTION

An augmented reality system may be configured to interact with objectswithin a scene and generate an augmented reality environment. Theaugmented reality environment allows for virtual objects and informationto merge and interact with tangible real-world objects, and vice versa.Disclosed herein are techniques and devices suitable for providingvarious interactions and functionality within the augmented realityenvironment.

The augmented reality system includes a computing device configured tosupport the augmented reality environment. Movement of real and virtualobjects may be tracked, along with a user's interactions with objects.Tracking of real objects allows for inventory management in theaugmented reality environment. Requests for an activity within theaugmented reality environment may then access this inventory informationand direct the activity accordingly. For example, a user request forhelp in planning and cooking a meal may be based on foodstuffs availablein the home at that time. Other uses for the inventory informationinclude generating replenishment orders for objects such as consumables.

By monitoring user interactions with objects in the augmented realityenvironment, various tasks are more easily performed. A pre-determinedgesture made towards a particular object may trigger placement of anorder for that object. Likewise, a pre-determined gesture made towards aparticular object may trigger an offer for transfer of property rightsin the object. The process of offering the object may include presentinga confirmation to the user to confirm the offering and possiblyparameters such as price, type of transfer, and so forth. It is thuspossible to allow users to easily buy or sell objects from the augmentedreality environment.

The awareness of objects in the augmented reality environment and theinteraction of one or more users with those objects also provides anadvertising presentation opportunity. As the user interacts with variousobjects in the environment, advertisements associated with the objectsmay be presented. For example, when a user picks up a beverage can, anadvertisement for a new flavor of cola may be presented. Details aboutthe interaction may be used to determine a level of interest in theadvertisement. Furthermore, upon issuing a pre-determined gesture, theuser may initiate purchase or additional inquiry about the goods orservice presented in the advertisement.

As described herein for illustration and not by way of limitation, theaugmented reality input and output may be provided at least in part by acombination of a structured light source, such as an image projectorconfigured to generate structured light patterns and a camera to imagethose patterns. The projector and camera may further be incorporatedinto a single unit and designated as an augmented reality functionalnode (ARFN). In other implementations, other combinations of elementssuch as projectors, cameras, microphones, ultrasound transducers, depthsensing cameras, weight sensors, touch sensors, tactile output devices,and so forth may be also be present within or coupled to the ARFN. Forconvenience, and not by way of limitation, the examples in thisdisclosure refer to the use of structured light for the characterizationof the physical environment of the scene including objects therein.However, in addition to or in place of structured light, othertechniques may be used such as light detection and ranging (LIDAR),optical time-of-flight, ultrasonic ranging, stereoscopic imaging, radar,and so forth either alone or in combination with one another.

Within the augmented reality environment, the ARFN includes an augmentedreality module. This module is configured to identify and track objectswithin the scene, maintain a user interface, and facilitate transactionsinvolving objects in the environment, as discussed in detail below.

Illustrative Environment

FIG. 1 shows an illustrative augmented reality environment 100 whichincludes one or more augmented reality functional nodes (ARFNs) 102(1),102(2), . . . , 102(N) with associated computing devices. In thisillustration, multiple ARFNs 102(1)-(N) are positioned in the corners ofthe ceiling of the room. In other implementations, the ARFNs 102(1)-(N)may be positioned in other locations within the scene. When active, onesuch ARFN 102 may generate an augmented reality environmentincorporating some or all of the items in the scene such as real-worldobjects. In some implementations, more or fewer ARFNs 102(1)-(N) may beused.

Each of the ARFNs 102(1)-(N) couples to or includes a computing device104. This computing device 104 may be within the ARFN 102, or disposedat another location and connected to the ARFN 102. The computing device104 comprises a processor 106, an input/output interface 108, and amemory 110. The processor 106 may comprise one or more processorsconfigured to execute instructions. The instructions may be stored inmemory 110, or in other memory accessible to the processor 106.

The input/output interface 108 may be configured to couple the computingdevice 104 to other components such as projector, cameras, microphones,other ARFNs 102, other computing devices, and so forth. The couplingbetween the computing device 104 and the devices may be via wire, fiberoptic cable, or wireless connection.

The memory 110 may include computer-readable storage media (“CRSM”). TheCRSM may be any available physical media accessible by a computingdevice to implement the instructions stored thereon. CRSM may include,but is not limited to, random access memory (“RAM”), read-only memory(“ROM”), electrically erasable programmable read-only memory (“EEPROM”),flash memory or other memory technology, compact disk read-only memory(“CD-ROM”), digital versatile disks (“DVD”) or other optical diskstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other medium which can be used tostore the desired information and which can be accessed by a computingdevice.

Several modules such as instructions, datastores, and so forth may bestored within the memory 110 and configured to execute on a processor,such as the processor 106. An operating system module 112 is configuredto manage hardware and services within and coupled to the computingdevice 104 for the benefit of other modules. Modules may be stored inthe memory of the ARFN 102, storage devices accessible on the localnetwork, or cloud storage accessible via a wide area network.

An object datastore 114 is configured to maintain information aboutobjects within or accessible by the augmented reality environment. Theseobjects may be tangible real world objects or virtual objects. Tangibleobjects include items such as tables, chairs, people, pets, foodcontainers, printed books, and so forth. Virtual objects include mediacontent such as songs, movies, electronic books, computer generatedscenes, media files, and so forth. Virtual objects may include storedcopies of those objects or access rights thereto. The object datastore114 may include a library of pre-loaded reference objects, as well asobjects which are temporally persistent within a particular environment,such as a wall, specific table, user and so forth. The object datastore114 or the other datastores described below may be stored on one or moreof the memory of the ARFN 102, storage devices accessible on the localnetwork, or cloud storage accessible via a wide area network. The objectdatastore 114 is discussed in more detail below in regards to FIG. 4.

A user interaction datastore 116 is configured to maintain informationabout interactions between one or more users and one or more objects.The user interaction data within the datastore 116 may be used in someimplementations to facilitate additional functions such as receiveindications of user selections of advertisements, provide usage data toproduct suppliers or manufacturers, and so forth. The collection and useof the user interaction data may be limited based at least in part uponthe preferences of the user. The user interaction datastore 116 isdiscussed in more detail below in regards to FIG. 5.

A transaction datastore 118 is configured to maintain information aboutchanges in property rights associated with objects. For example, thedatastore 118 may be configured to track the purchase, rental, sale,lease, and so forth of objects. The transaction datastore 118 isdiscussed in more detail below in regards to FIG. 6.

An advertisement datastore 120 is configured to maintain informationabout advertisements available to the augmented reality environment. Theadvertisement datastore 120 is discussed in more detail below in regardsto FIG. 7.

An augmented reality module 122 is configured to generate augmentedreality output in concert with the physical environment. The module 122may access one or more of the datastores described herein. A trackingand control module 124 is configured to identify objects includingusers. This identification may include the use of a camera, structuredlight, radio frequency identification equipment, and so forth within theARFN 102. Objects may be identified by shape, text thereon, universalproduct code (UPC), optical barcode, radio frequency identification tag(RFID), and so forth.

The term object encompasses several categories including unitary,package, group, container, and so forth. Some objects may be unitary inthat they are complete in and of themselves, such as a baseball bat orwrench. Some objects may be packages containing a plurality of itemswhich may or may not be identical, such as a twelve-can case of cola orset of different sized wrenches. An object may also be a group ofclosely interrelated sub-objects, which operate together to form awhole, such as a jigsaw puzzle. An object may also be a containerholding other objects, such as a filled tote with the significance beingthe contents of the tote and not necessarily the container itself.

The category of an object may be determined at least in part by factorsincluding context, prior use, pre-determined data, and so forth. Forexample in the context of home use and purchase, wine may be considereda unitary object and designated by the bottle not by the case.Similarly, pre-determined data may indicate that totes are used forstorage, and as such a user pointing to a tote filled with objects maybe determined as a default to be designating the objects within thetote, and not the tote itself.

A user interface module 126 is configured to accept and interpret inputand generate output for the user. The ARFN 102 may use a camera,structured light, stereoscopic vision, and so forth to read the inputfrom the user. A transaction module 128 is configured to associateobjects with advertisements, facilitate transactions involving thirdparties such as buying and selling objects, and perform other functions.

FIG. 2 shows an illustrative schematic 200 of one example augmentedreality functional node 102 and selected components. The ARFN 102 isconfigured to scan at least a portion of a scene 202 and the objectstherein. The ARFN 102 may also be configured to provide augmentedreality output, such as images, sounds, and so forth.

A chassis 204 holds the components of the ARFN 102. Within the chassis204 may be disposed a projector 206 that generates and projects imagesinto the scene 202. These images may be visible light images perceptibleto the user, visible light images imperceptible to the user, images withnon-visible light, or a combination thereof. This projector 206 may beimplemented with any number of technologies capable of generating animage and projecting that image onto a surface within the environment.Suitable technologies include a digital micromirror device (DMD), liquidcrystal on silicon display (LCOS), liquid crystal display, 3LCD, and soforth. The projector 206 has a projector field of view 208 whichdescribes a particular solid angle. The projector field of view 208 mayvary according to changes in the configuration of the projector. Forexample, the projector field of view 208 may narrow upon application ofan optical zoom to the projector. In some implementations, a pluralityof projectors 206 may be used.

A camera 210 may also be disposed within the chassis 204. The camera 210is configured to image the scene in visible light wavelengths,non-visible light wavelengths, or both. The camera 210 has a camerafield of view 212 which describes a particular solid angle. The camerafield of view 212 may vary according to changes in the configuration ofthe camera 210. For example, an optical zoom of the camera may narrowthe camera field of view 212. In some implementations, a plurality ofcameras 210 may be used.

The chassis 204 may be mounted with a fixed orientation, or be coupledvia an actuator to a fixture such that the chassis 204 may move.Actuators may include piezoelectric actuators, motors, linear actuators,and other devices configured to displace or move the chassis 204 orcomponents therein such as the projector 206 and/or the camera 210. Forexample, in one implementation the actuator may comprise a pan motor214, tilt motor 216, and so forth. The pan motor 214 is configured torotate the chassis 204 in a yawing motion. The tilt motor 216 isconfigured to change the pitch of the chassis 204. By panning and/ortilting the chassis 204, different views of the scene may be acquired.The spatial analysis module 114 may use the different views to monitorobjects within the environment.

One or more microphones 218 may be disposed within the chassis 204, orelsewhere within the scene. These microphones 218 may be used to acquireinput from the user, for echolocation, location determination of asound, or to otherwise aid in the characterization of and receipt ofinput from the scene. For example, the user may make a particular noise,such as a tap on a wall or snap of the fingers, which are pre-designatedas attention command inputs. The user may alternatively use voicecommands. Such audio inputs may be located within the scene usingtime-of-arrival differences among the microphones and used to summon anactive zone within the augmented reality environment.

One or more speakers 220 may also be present to provide for audibleoutput. For example, the speakers 220 may be used to provide output froma text-to-speech module or to playback pre-recorded audio.

A transducer 222 may be present within the ARFN 102, or elsewhere withinthe environment, and configured to detect and/or generate inaudiblesignals, such as infrasound or ultrasound. These inaudible signals maybe used to provide for signaling between accessory devices and the ARFN102.

A ranging system 224 may also be provided in the ARFN 102. The rangingsystem 224 is configured to provide distance information from the ARFN102 to a scanned object or set of objects. The ranging system 224 maycomprise radar, light detection and ranging (LIDAR), ultrasonic ranging,stereoscopic ranging, and so forth. In some implementations thetransducer 222, the microphones 218, the speaker 220, or a combinationthereof may be configured to use echolocation or echo-ranging todetermine distance and spatial characteristics.

In this illustration, the computing device 104 is shown within thechassis 204. However, in other implementations all or a portion of thecomputing device 104 may be disposed in another location and coupled tothe ARFN 102. This coupling may occur via wire, fiber optic cable,wirelessly, or a combination thereof. Furthermore, additional resourcesexternal to the ARFN 102 may be accessed, such as resources in anotherARFN 102 accessible via a local area network, cloud resources accessiblevia a wide area network connection, or a combination thereof.

Also shown in this illustration is a projector/camera linear offsetdesignated “O”. This is a linear distance between the projector 206 andthe camera 210. Placement of the projector 206 and the camera 210 atdistance “O” from one another aids in the recovery of structured lightdata from the scene. The known projector/camera linear offset “O” mayalso be used to calculate distances, dimensioning, and otherwise aid inthe characterization of objects within the scene 202. In otherimplementations the relative angle and size of the projector field ofview 208 and camera field of view 212 may vary. Also, the angle of theprojector 206 and the camera 210 relative to the chassis 204 may vary.

In other implementations, the components of the ARFN 102 may bedistributed in one or more locations within the environment 100. Asmentioned above, microphones 218 and speakers 220 may be distributedthroughout the scene. The projector 206 and the camera 210 may also belocated in separate chassis 204. The ARFN 102 may also include discreteportable signaling devices used by users to issue command attentioninputs. For example, these may be acoustic clickers (audible orultrasonic), electronic signaling devices such as infrared emitters,radio transmitters, and so forth.

FIG. 3 illustrates an environment 300 encompassing the computing device,users, advertisement servers, merchants, and so forth. Users 302(1),302(2), . . . 302(N) may use the computing device 104 in the augmentedreality environment or other computing device 304, such as desktopcomputers, to access resources via a network 306. The network 306 mayinclude the internet, private network, local area network, and so forth.

Resources available via the network 306 may include advertisementservers 308, marketplace transaction servers 310, online merchantservers 312, bricks and mortar merchant servers 314, cloud resources316, and so forth. The advertisement servers 308 are configured toprovide advertisements for presentation within the augmented realityenvironment. The marketplace transaction servers 310 facilitatetransactions such as buying or selling objects. For example, themarketplace transaction servers 310 may provide a virtual marketplaceallowing for users 302 to buy and sell used objects.

Online merchant servers 312 provide online storefronts and the abilityto select and purchase objects. Bricks and mortar merchant servers 314may be coupled with point of sale devices such as cash registers toprovide sales data 318 to the augmented reality environment.

The computing device 104 may also access cloud resources 316 via thenetwork 306. These cloud resources 316 may include additional storage,processing capabilities, application access, and so forth.

FIG. 4 is an illustrative diagram of an object datastore 114 configuredto store information about objects in the augmented reality environment.As described above, these objects may be tangible real world objects orvirtual objects.

Information about the object may include specific object information 402which describes a particular object. This may include a uniqueidentifier 402(1), date acquired, 402(2), expiration date 402(3),uniqueness flag 402(4), and so forth. For example, the objectinformation 402 may define that an object with serial number 12345 wasacquired on Apr. 1, 2011, will expire on Apr. 1, 2013, and is a fungiblenon-unique item. Object attributes 404 may also be stored. Theseattributes may include one or more categories 404(1), description404(2), ownership rights 404(3), status 404(4), purchase price 404(5),current value 404(6), and so forth. For example, the object 12345 may bein the categories 404(1) of food, canned, and fruit with a description404(2) of stewed prunes.

The ownership rights 404(3) may indicate what property rights are heldby a particular user or group of users. For example, ownership rights404(3) may include possession but not ownership if the object is beingleased or rented from another. Continuing the example, the ownershiprights 404(3) may indicate that the user holds all property rights inthe can of stewed prunes and the status 404(4) may indicate that the canhas been partially consumed. A purchase price 404(5) of $1.95 may berecorded, and the current value of the remaining contents 404(6) may be$0.37.

Physical attributes 406 about the object may also be stored in theobject datastore 114. The physical attributes 406 may includesize/physical dimensions 406(1), color 406(2), weight 406(3), topology406(4), and so forth. For example, the can of stewed prunes may be 300mm tall and 50 mm in diameter, be predominately white, weigh 700 grams,and be generally cylindrical in shape.

Location 408 information about the object may also be maintained. Thismay include a current location 408(1), default location 408(2),associated locations 408(3), and so forth. Continuing the above example,the can of stewed prunes may be located on the second shelf of therefrigerator, but have a default location of the fourth shelf in thepantry. The stewed prunes may also be associated with locations such asthe countertop, stove, and dining room. The location 408 may bespecified as an absolute physical location within the augmented realityenvironment or a relative physical location. For example, an absolutephysical location may be that the stewed prunes are located atcoordinates 3400 mm, −975 mm, 65 mm in a Cartesian coordinate systemdescribing the augmented reality environment encompassing the room. Incontrast, an example of a relative physical location would be that thestewed prunes are located in the kitchen.

A quantity 410 of objects present in the augmented reality environmentmay also be stored. These quantities may be associated with particularlocations 408, or maintained for the augmented reality environment as awhole. Quantity thresholds 412 may also be stored. For example, a usermay specify that they prefer to have at least three cans of stewedprunes in stock but no more than five. The quantity thresholds 412 mayalso be specified by location. For example, the user may wish to haveonly a single can of stewed prunes in the refrigerator at a time.Maintaining data about the location, quantity, and so forth, allows theuser to access this information to make choices or take actions withinthe environment which involve those objects.

FIG. 5 is an illustrative diagram of a user interaction datastore 116configured to store information about how users interact with objects inthe augmented reality environment. The datastore 116 may include thespecific object information 402 to associate particular interactionswith a particular object or group of objects.

A user identification 502 may be stored, indicating which user or usersinteracted with the object. For example, the user identification 502 mayindicate that the user 302(1) interacted with the can of stewed prunes.The user interaction datastore 116 may also store a date/time 504 of theinteraction, a nature of interaction 506, and a duration 508 of theinteraction. Continuing the example, the date/time of interaction withthe can of stewed prunes was on Apr. 1, 2011 at 8:37:10, the nature ofthe interaction was picking up the can, and the duration was 22 seconds.

The user interaction datastore 116 may also store location information510 about the interaction. This location information may include astarting location 510(1), end location 510(2), and so forth. In ourexample, this may indicate that the starting location of the interactionwith the canned prunes was the second shelf of the refrigerator and theend location was the kitchen counter.

Users or processes may query the user interaction datastore 116. Forexample, the user 302 may query to determine who last ate the stewedprunes, whether the prunes were put away, and so forth. In anotherexample, the user 302 may query to determine the location for amisplaced set of car keys.

The user interaction datastore 116 may also be queried to generate alisting of objects which are frequently or infrequently used. Forexample, when the user 302 is preparing to move to a new apartment hemay wish to dispose of unused and unwanted objects. The user 302 mayquery the user interaction datastore 116 to list all objects which usershave not interacted with in the previous twelve months.

FIG. 6 is an illustrative diagram of a transaction datastore 118. Thetransaction datastore 118 is configured to store information abouttransactions involving objects in the augmented reality environment. Thespecific object information 402 and user identification 502 may beassociated with transactional details such as transaction date/time 602,transaction type 604, quantity of the transaction 606, relatedtransactions 608, special conditions of the transaction 610, buy details612, sell details 614, and so forth.

When the transaction type 604 indicates the transaction involves apurchase of an object, the buy details 612 may be captured. This datamay include purchase price 612(1), merchant identification 612(2),delivery details 612(3), and so forth.

In some situations, such as when the transaction type 604 indicates thetransaction involves a sale, the sell details 614 may be populated. Thedetails may include a sale price 614(1) indicating a sale price of theobject, a buyer identification 614(2) indicating the identity of abuyer, delivery details 614(3) describing how the object is to bedelivered, and other information.

FIG. 7 is an illustrative diagram of an advertisement datastore 120configured to store information about advertisements associated withobjects. The advertisement comprises a creative 702 and other detailssuch as found in the advertisement datastore 120. The creative 702 maycomprise audio, video, text, image, or other media. The creative 702 inthe datastore 120 may be a copy of an advertisement or may comprise alink or reference to another storage location. For example, the creative702 may be a video clip describing the benefits of a virtual gym withpersonal trainers. Connections to or copies of associated creatives 704may also be stored. For example, the creative 702 of the advertisingpersonal training services may have associated creatives 704 pertainingto free weights, exercise clothing, diet services, and so forth.

Associated objects 706 describe what objects are associated with theadvertisement. For example, a barbell may be associated with anadvertisement for personal trainer services.

Associated categories 708 indicate what categories of objects areassociated with the advertisement. For example, the object categories ofexercise equipment, health and fitness, weight loss, and so forth may beassociated with the advertisement for personal trainer services.

Advertisements may be associated with particular users or groups ofusers. Associated users 710 may be designated which are consideredsuitable for the particular creative 702. For example, the ad forpersonal trainer services may be associated with adult users, but notadolescents. Therefore, in this example, when the environment recognizesan adult has entered the room containing the barbell, an ad for personaltrainer services may be presented. In contrast, when the environmentrecognizes a child has entered the room, an ad for eating healthy snacksmay be presented instead.

The immersive nature of the augmented reality environment also providesopportunity for targeted location aware advertisements. Associatedlocations 712 indicate locations, absolute or relative, in which theadvertisement 120 is considered relevant. For example, an advertisementfor personal trainer services may be presented when the user is in aroom with exercise equipment, but not when in the kitchen.

Presentation parameters 714 may also be stored. These parameters mayinclude a user prompt type 714(1), start date/time 714(2), stopdate/time 714(3), frequency 714(4), and association thresholds 714(5).The user prompt type 714(1) indicates whether the user prompt is ahighlight of an object, full motion video projection, audio prompt, andso forth. The time when the creative 702 is presented may be specifiedwith the start date/time 714(2), stop date/time 714(3) and frequency.For example, the creative 702 is to be presented at most five timesbetween 16:00 and 20:00 every day.

The association threshold 714(5) indicates what level of involvementshould exist between the user and the object before the creative isassociated for presentation. The level of involvement may indicate aphysical proximity or contact, such as an instruction for the system toassociate the creative with the object when the user touches orapproaches the object. Continuing the example above, the creative aboutthe personal trainer service may be configured with the associationthreshold 714(5) set such that the ad is presented when the user reachesfor the barbells, but not if the user faces them.

FIG. 8 is a schematic of a user interface module 126. As describedabove, the module 126 is configured to accept and interpret input andgenerate output for the user. The user interface module 126 may includea gesture recognition module 802 configured to recognize gestural input.A gesture is a sequence of motions made by the user or an object underthe user's control within the augmented reality environment. Thesemotions need not be in contact with a touch sensing input device. Forexample, a user may draw a finger in a particular pattern across thesurface of a table or wall. The camera 210 may detect this motion anddetermine the motions of the user.

In some implementations, the gesture may occur at least in part freefrom contact with other objects. For example, the gesture may comprisethe user extending a pointer finger and touching an object, or viceversa. The gesture need not, but may, call for contact. For example, thegesture may include the user waving a hand, pointing at an object,arranging their fingers in a particular configuration, and so forth.

Some gestures may also include audible elements, such as raising a handand generating a finger snap, or rapping on a wall. Input may alsoinclude speech accepted by a speech recognition module 804. Other formsof input may also be accepted from input devices including keyboards,buttons, pointers, touch sensors, and so forth.

A user prompt module 806 in the user interface module 126 is configuredto generate user prompts such as highlights, pointers, menus, icons, andso forth. For example, the user prompt module 806 may be configured topresent a menu of options to the user. The user prompt module 806 maygenerate audible, visual, tactile, or other outputs.

FIG. 9 is a schematic of a transaction module 128. An advertisementmodule 902 is configured to access data from the advertisement datastore120 and associate objects in the augmented reality environment withadvertisements. This association may be based upon the factors describedabove with regards to the advertisement datastore 120, such as theassociated objects 706, associated categories 708, associated users 710,associated locations 712, and so forth.

A buy module 904 is configured to process user requests in the augmentedreality environment to purchase objects. A sell module 906 is likewiseconfigured to process user requests in the augmented reality environmentto sell objects. These modules may access a transaction interface module908 configured to provide transactional information to others such asmerchants, other users, and so forth. For example, the advertisementmodule 902 may find an associated creative 702 and call the userinterface module 806 to present the creative 702 to the user. A positiveuser response to the ad may result in the buy module 904 generating anorder for the good or service presented in the ad. The transactioninterface module 908 then provides the order to the appropriatesupplier. An order acknowledgement may be received, and then presentedto the user.

Illustrative Processes

Described next is an augmented reality environment within a room, suchas may be generated with the ARFNs 102(1)-(N). Following the descriptionof this environment, several example processes are provided with respectto this augmented reality environment.

FIG. 10 is a detailed perspective view of an illustrative augmentedreality environment 1000, such as may be maintained by the ARFNs102(1)-(N). In this view a floor 1002, walls 1004, and ceiling 1006 of aroom are shown. A user 1008 is standing on the floor 1002 of the room.While a single user 1008 is shown, it is understood that the augmentedreality environment may support multiple users. Several ARFNs 102(1)-(N)are shown mounted on the ceiling 1006. While four ARFNs 102 are shownmounted in the approximate corners of the ceiling 1006, more or fewerARFNs 102 may be used in other implementations, and such ARFNs may bedistributed to other locations of the ceiling 1006, on the walls 1004,tables, and so forth.

Disposed on one of the walls 1004 is a television 1010 or computerdisplay. This television/display 1010 may present at least a portion ofthe augmented reality environment. In some implementations the computerdisplay may be a portable electronic device, such as a tablet computer.On a wall is a painting 1012, while a refrigerator 1014 sits in acorner. The refrigerator 1014 may incorporate a tag scanner 1016. Thisscanner 1016 may be configured to read tags or labels of items placedinto or removed from the refrigerator or another location. The tagscanner 1016 may comprise a barcode reader, radio frequencyidentification tag, camera, and so forth. In some implementations, thetag scanner 1016 may be a separate unit which is moveable within theaugmented reality environment. Sitting in another corner of the room isa barbell 1018. In another corner is a table 1020 with a beverage can1022 resting thereon.

FIG. 11 is an illustrative process 1100 of tracking user interactions inan augment reality environment, such as the augmented realityenvironment 1000 described above. The processes described in thisdisclosure may be implemented by the architectures described herein, orby other architectures. These processes are illustrated as a collectionof blocks in a logical flow graph. Some of the blocks representoperations that can be implemented in hardware, software, or acombination thereof. In the context of software, the blocks representcomputer-executable instructions stored on one or more computer-readablestorage media that, when executed by one or more processors, perform therecited operations. Generally, computer-executable instructions includeroutines, programs, objects, components, data structures, and the likethat perform particular functions or implement particular abstract datatypes. The order in which the operations are described is not intendedto be construed as a limitation, and any number of the described blockscan be combined in any order or in parallel to implement the processes.It is understood that the following processes may be implemented onother architectures as well.

At 1102, a user is identified in an augmented reality environment. Asdescribed above, the tracking and control module 124 may perform theidentification. This identification may be absolute or relative. Asshown here, the augmented reality system may absolutely recognize a useras user 102(1), potentially with reference to the user identification502 information described above. Also shown is a particular access levelof “administrator” associated with the user 102(1). In comparison toabsolute identification, a relative identification is that person A andperson B are different individuals but with unknown identities.

At 1104, the user provides input approving acquisition of userinteraction data in the augmented reality environment. In someimplementations, this input may be via the projector 206 and the camera210 of the ARFN 102 emitting and receiving, respectively, structuredlight and the tracking and control module 124 processing this input. Forexample, a user may make a gesture of the sign language “w” associatedwith a mnemonic to allow the ARFN 102 to “watch” what is happening inthe augmented reality environment. The ability to approve acquisition ofdata may be limited to specific users or groups of users. For example,an adult user may have authority to approve acquisition of user data,while a child may not.

At 1106, acquired user interaction data is maintained within a userinteraction datastore 116. For example, as shown here the userinteraction datastore 116 has stored that user 102(1) for a specificperiod of time has consumed five colas, exercised, listened to somemusic, and otherwise interacted with various objects and functions inthe augmented reality environment.

At 1108, user input is accepted which approves distribution of a least aportion of the interaction data from the user interaction datastore 116.For example, the user may make a gesture indicating that the augmentedreality system is allowed to share cola consumption with an advertisingcompany.

At 1110, at least a portion of the user interaction data from the userinteraction datastore 116 is distributed to another party. For example,the computing device 104 then provides the cola consumption data to theadvertisement servers 308 via the network 306. In some implementations,the transaction module 128 may perform the distribution.

FIG. 12 is an illustrative process 1200 of inventory management in anaugmented reality environment. Because the augmented reality environmentmay be configured to recognize objects as well as their locations, theaugmented reality system is able to provide inventory trackingfunctions.

At 1202, a volume within the augmented reality environment is designatedas an inventory management volume. This designation may be performed bythe tracking and control module 124. For example, this may be aparticular appliance such as a refrigerator, location such as acupboard, room such as the kitchen, and so forth.

At 1204, tracking data comprising movement of objects with regard to theinventory management volume is maintained. For example, the user mayplace a can of stewed prunes into the refrigerator 1014 and remove twoboxes of granola bits, updating the inventory of objects in therefrigerator 1014. Other details about the objects may be tracked aswell, such as expiration dates, replacement periods, quantity used, andso forth. This tracking may be performed by the tracking and controlmodule 124. In one implementation, the tag scanner 1016 may be used toprovide tracking data of object movement.

At 1206, based at least in part upon the tracking data, inventory datamay be provided to the object datastore 114. In this example, the objectdatastore 114 may be provided with the data that the stewed prunes arenow in the refrigerator 1014 while the granola bits have been removedfrom the refrigerator 1014. The inventory of the refrigerator contentsmay then be adjusted to reflect that there are now three cans of stewedprunes and no granola bits.

FIG. 13 is an illustrative process 1300 of assisting an activity in anaugmented reality environment. At 1302, a request for an activity withinthe augmented reality environment 1000 is received, such as by thetracking and control module 124. Activities may include cooking,crafting, games, maintenance, repairs, procedures, and so forth. Forexample, when as shown here the augmented reality environment existswithin a home the user may ask “what can I make for dinner for threepeople?”

At 1304, the object datastore 120 is queried for materials associatedwith the activity and availability of those materials. In someimplementations, the tracking and control module 128 may generate thequery. Continuing the example, an inventory of foodstuffs which areavailable for consumption in the object datastore 114 is compared to arecipe database accessed via the network 306. Where a list of thematerials associated with the activity is not available, the system mayprompt the user for the list.

At 1306, output related to use of the materials associated with theactivity within the augmented reality environment 1000 is generated. Asshown here, a suggested menu for tonight serving three people and usingfoodstuffs which are on hand is presented. Should the user request it,the augmented reality system may provide additional prompts and detailsas to where the necessary foodstuffs are located, details on how toprepare them, and so forth.

FIG. 14 is an illustrative process 1400 of receiving objects into anaugmented reality environment and replenishing those objects. Objectsmay be received from a variety of sources, such as deliveries frommerchants. In such a situation, manifest data may be received from themerchant and used to receive objects into the augmented realityenvironment 1000. The transaction module 128 may act to receive theinformation from the merchant, while the tracking and control module 124determines the physical arrival or movement of objects in the augmentedreality environment.

At 1402, an order for objects to a supplier is provided from theaugmented reality environment 1000. The transaction interface module 908may communicate the order via the network 308 to the online merchantserver 312. For example, an order may be placed with online merchant“Danube” for two cases of Acme Cola.

At 1404, objects are physically received at the augmented realityenvironment such as a user's home and an update is made to the objectdatastore 114 based at least in part upon data associated with theorder. Continuing the example, the manifest for the shipment made by theonline merchant “Danube” shows a quantity of two cases of Acme Cola.Upon confirmation of delivery at the customer, the object datastore 114associated with the customer is updated to reflect the increase ininventory level of Acme Cola.

A delivery carrier may provide the confirmation of delivery, theaugmented reality system may recognize the parcel containing theshipment, or a combination thereof may also be used. In oneimplementation, the manifest may encoded within the physical shipment inmachine readable form. For example, a radio frequency identification tagmay store the manifest. The tracking and control module 124 may then beconfigured to read this tag and process the information stored therein.In another implementation, the manifest may be transmitted from thesupplier or shipper to the augmented reality system via the network 306.

At 1406, the object datastore 114 is adjusted based at least in partupon user interaction data within the user interaction datastore 116.For example, the user may place three cans of Acme Cola in a finallocation designated as a trash can. Following placement of the cans intothe trash can, the inventory level of Acme Cola is decremented in theobject datastore 114.

At 1408, a recommended replenishment order based at least in part uponthe object datastore 114 information is provided. The quantity 410 onhand and the quantity thresholds 412 may be used to affect when thereplenishment order is provided. Here, the user is shown a recommendedorder showing several items which are below pre-determined replenishmentthresholds, such as Acme Cola and granola bits. These pre-determinedreplenishment thresholds may differ from object to object. For example,Acme Cola may have a replenishment threshold of twelve cans whilegranola bits may have a replenishment threshold of two packages.

The recommended replenishment order may include substitutions forobjects, supplier, shipping/delivery method, and so forth. For example,the recommended replenishment order may suggest Coyote Cola rather thanAcme Cola. Determination of what to substitute and whether to presentthis substitution to the user may be made based on parameters defined bythe user, the system 100, a supplier, an advertiser, or a combinationthereof. For example, the user may have set a user parameter calling forpresentation of lower priced alternatives in replenishment orders. Thus,when Coyote Cola goes on sale it may be suggested within thereplenishment order. In some implementations, the substitution may bemade without presentation to the user.

At 1410, the replenishment order is provided to the supplier. Forexample, the transaction interface module 908 may connect via thenetwork 308 to transmit the order to the online merchant servers 312 forthe supplier “Danube.”

A confirmatory prompt may be presented prior to providing the order tothe supplier. For example, the user interface module 126 may beconfigured to present a listing of the order via the projector 206 on awall in the user's line of sight. In another implementation, thereplenishment order may be sent automatically. As shown here, the orderincluding the Acme Cola is placed with the online merchant servers 310,as described above.

FIG. 15 is an illustrative process 1500 of initiating a purchase in theaugmented reality environment. The capability of the augmented realitysystem to receive gestures and other inputs of users allows for newmechanisms for initiating purchases. With the techniques and processesdescribed herein, it becomes possible to purchase with a single touch ofan object or a gesture having one or more motions which occur at leastin part free from contact with other objects.

At 1502, a pre-determined gesture is received in an augmented realityenvironment. The pre-determined gesture may have a mnemonic connotation,such as the American Sign Language letter “b” for “buy.” This gesturemay comprise movements, speech, or a combination thereof.

At 1504, the pre-determined gesture is associated with an object in theaugmented reality environment. For example, it may be an object theuser's gesturing hand is pointed to, or an object which the user isholding, touching, or within a pre-determined distance of. As describedabove, these objects may be tangible real world objects or virtualobjects. Tangible objects include items such as tables, chairs, people,pets, food containers, printed books, and so forth. Virtual objectsinclude media content such as songs, movies, electronic books, computergenerated scenes, and so forth. As described above, the object itselfmay be categorized as a unitary item, package, group of closelyinterrelated objects, container, and so forth.

As also described above, the category of object used may be determinedat least in part by factors including context, prior use, pre-determineddata, and so forth. For example, as shown here where the single can ofCola is sitting on a table, in the context of generating an order theselected object is a package object. In contrast, in the context of theuser requesting nutritional information about the Cola the nutritionalinformation about the number of calories in a single can is presentedrather than the number of calories in the entire package.

In another implementation, the user may issue a pre-determined gestureor other command to designate the category of the object. For example, auser may point to a package of Cola then issue a gesture indicating thenumber “1” which the system may then interpret as meaning select fromthe Cola a single can.

Association with the object may be dependent upon the context of theinteraction. For example, suppose the user is watching a broadcast showon the television 1010. During a commercial break in the broadcast show,an advertisement for a particular song is presented on the television1010. The user may point to the screen and make the pre-determinedgesture. Within the environmental context at the time of the gesture,the augmented reality system determines that the user is not intendingto purchase another television set 1010, but rather is attempting to buythe particular song being advertised on the television 1010 at the timeof the gesture.

At 1506, a selection indicator designating the object is generated. Thisselection indicator may comprise a visual indicator proximate to oroverlapping the associated object. For example, a ring, arrow, overlay,audible prompt and so forth may be used to indicate the associatedobject. In other implementations, audible prompts may also be used inconjunction with the visible prompts. These audible indicators may beconfigured to have an apparent source proximate to the associatedobject. For example, an audible prompt may be perceived by the user tocome from (or nearby) the object.

The selection indicator may have several attributes such as size,brightness, color, intensity, shape, image, audio waveform, and soforth. One or more of these attributes may be modified according tocharacteristics of the object. For example, the one or more attributesof the selection indicator may be modified in proportion to the purchaseprice 612 of the associated object. Thus, an inexpensive item may have apale green selection indicator while an expensive item may have a brightred selection indicator. Or the selection indicator may have one ringaround the object for every ten dollars (rounded upwards) of purchaseprice, thus a seven dollar object would have a single ring and athirty-nine dollar object would have four rings.

The selection indicator designating the object may also be modifiedbased at least in part upon local availability. For example, when theavailable inventory as maintained in the object datastore 114 indicatesthat the associated object such as the Acme Cola is available in thehome, the selection indicator may change shape, or include a trail ofarrows pointing to the physical location where the extra Acme Cola isstored in the pantry.

In some implementations, a set of possible suppliers may be determinedand presented to the user. For example, the selection indicator mayinclude icons or trademarks associated with suppliers which provide thatobject. As a result, the user may see the selection indicator ring anddisposed around that ring are icons for store “A”, store “B” and store“C”. The user may then point to the particular supplier he wants toorder from. In some implementations, a particular gesture or portion ofa gesture may be associated with a particular supplier.

At 1508, an order for the associated object is generated. A quantity ofthe order may be set to a default, or be adjusted based upon thegesture. A gesture indicating a numerical quantity or a magnitude of thepre-determined gesture may be used. For example, a “b” hand sign with ashort arc may indicate buy a quantity of one, while a longer arc mayindicate a quantity of two. In another implementation, the quantity maybe dynamically adjusted after comparison to the quantity 410 and thequantity thresholds 412 in the object datastore 114. Thus, apre-determined gesture to buy may trigger an order for a quantitynecessary to replenish the inventory of the object up to a maximumquantity threshold. At least a portion of the order may be stored in thetransaction datastore 118.

The system may identify a user issuing the pre-determined gesture anddetermine an authorization level of the user. The ability to generate anorder may be limited to particular users or groups of users. The usermay be identified by the augmented reality system and assigned orderrights. These order rights may be restricted to particular categories ofitem, price limits, and so forth. For example, authorized adults may beable to purchase anything, while children may be limited to purchasingonly school-related objects.

In some implementations, the user may be authorized for the purchasebefore the order for the associated object is generated. Orders from oneor more unauthorized users may also be queued for confirmation by anauthorized user. For example, orders initiated by a child or houseguestmay be held in a queue for a parent to review.

In some implementations, prior to generating the order for theassociated object, alternative options or substitutions may be presentedto the user for selection. These substitutions may include alternativeor comparable objects, suppliers, shipping/delivery methods, and soforth. The user may be presented with options to substitute Acme Colawith Birch Cola, purchase a 24 pack instead of a twelve pack, and soforth.

At 1510, the order is provided to a supplier of the object. The suppliermay be another user, a commercial merchant, and so forth. The order or aportion thereof may be provided from the computing device 104 via thenetwork 306 to the supplier. In some implementations, a plurality oforders may be queued for a pre-determined period prior to providing theorders to the supplier. For example, orders may be queued for four hoursprior to providing the orders to the merchant to minimize loads on themerchant servers and encourage order consolidation. In otherimplementations, the queued orders may be consolidated into a singleorder.

FIG. 16 is an illustrative process 1600 of initiating an offer fortransfer of property rights in an augmented reality environment. Thetransfer of property rights may include a gift, donation, sale, lease,rental, creation of a lien, creation of a mortgage, and so forth for aparticular object. As mentioned above, the object may be a physicalobject or a digital object.

At 1602, a pre-determined gesture is received in an augmented realityenvironment. As described above, the pre-determined gesture may have amnemonic connotation, such as the American Sign Language letter “s” for“sell.” This gesture may comprise movements, speech, or a combinationthereof.

At 1604, the pre-determined gesture is associated with an object in theaugmented reality environment. For example, it may be an object theuser's gesturing hand is pointed to, such as the painting 1012 shownhere. In other implementations the gesture may be associated to anobject which the user is holding, touching, or within a pre-determineddistance of. As described above, these objects may be tangible realworld objects or virtual objects. Tangible objects include items such astables, chairs, food containers, printed books, and so forth. Virtualobjects include media content such as songs, movies, electronic books,computer-generated scenes, and so forth. Association with the object maybe dependent upon the context of the interaction.

As described above, in some implementations the system 100 may provide arecommendation of what objects to offer. In such an implementation, theuser gesture may initiate a query of the user interaction datastore 116as described above for objects. The user may select query parameters,such as the period of time unused, maximum value, size of the object,and so forth. Data about gesture itself may also be used to set a queryparameter. For example, a user may make very wide hand gesturesindicating the entire premises, in which case the query would includeobjects throughout the accessible physical volume. The object(s)selected by the query may then be associated with the gesture.

At 1606, a selection indicator designating the object is generated. Thisselection indicator may comprise a visual indicator proximate to oroverlapping the associated object. For example, a ring, arrow, overlay,audible prompt and so forth may be used to indicate the associatedobject. As shown here, a border is projected around the painting 1012 onthe wall.

As described above with regards to FIG. 15, the selection indicator mayhave several attributes such as size, brightness, color, intensity, andso forth. One or more of these attributes may be modified according tocharacteristics of the object. In some implementations, one or moreattributes of the selection indicator may be modifies in proportion tothe offered sale price of the associated object.

The selection indicator attributes may also be modified to indicate whenthe local availability of the associated object exceeds a pre-determinedthreshold. For example, where the object datastore 114 indicates theuser has twenty-five identical objects and the user has a pre-determinedthreshold of keeping at least five, the selection indicator may be greenindicating the user has plenty to sell. However, should the objectdatastore 114 indicate that only five objects are in stock, theselection indicator may turn red indicating the user will be going belowthe pre-determined threshold should the sale proceed.

The attributes of the selection indicator may also be modified inproportion to a proposed purchase price of the associated object. Forexample, where the associated object is contemporaneously listed asbeing wanted by someone else, the proposed purchase price may affect theselection indicator.

At 1608, an offer for the transfer of property rights for the associatedobject is generated. Property rights of the object may be determined, aswell as property rights of the object relative to a particular user. Forexample, the sell module 906 of the transaction module 128 may accessthe ownership rights 404(3) in the object datastore 114 to determine ifthe object can be sold and, if so, whether ownership rights allow aparticular user to sell the object. This prevents a user from conveyingproperty rights in an object which they may not have. As shown here, anoffer has been generated for the painting 1012 indicating the quantity,description, artist, and sale amount.

Prior to generating the offer, the identity of the user issuing thepre-determined gesture may be determined as well as an authorizationlevel associated with the user. For example, a child may not beauthorized to sell an appliance in the house.

An appraisal may also be generated. This appraisal comprises an estimateor projection of a value associated with a transfer of one or moreproperty interests associated with an object. For example, the system100 may be configured to provide an assessment of the value of theselected painting 1012. The user may then use this appraised value toset the offer price for the painting 1012. This appraisal may bedependent upon the property rights transferred. Continuing the example,an appraisal of an offer price for sale may be different from anappraisal for a six-month rental.

At 1610, the offer is provided via the network 306 to another party,merchant, marketplace, and so forth. In other implementations, an imageof the associated object may be acquired by the ARFN 102 and included inthe offer. Other data may also be provided in the offer, including thedate acquired 402(2), the purchase price 404(5), and so forth. Asdescribed above, offers from one or more unauthorized users may bequeued for confirmation by an authorized user before providing them toanother party.

In some implementations, shipping materials are provided to the user302. The marketplace, a shipping company, and so forth may provide theshipping materials. The shipping materials may be provided when theoffer is made or after acceptance of the offer. Shipping materialsinclude shipping containers, boxes, padding, shipping labels, and soforth. For example, the sell module 906 may be configured to ordershipping materials for delivery to the user 302.

FIG. 17 is an illustrative process 1700 of presenting advertisements orother notifications in an augmented reality environment. At 1702, anobject within an augmented reality environment is identified. Forexample, as shown here the augmented reality system has identified thebarbell. In some implementations, attention of a user with regards tothe object may also be considered when determining which object is to beidentified in the environment for presentation of an advertisement.Attention may be determined by analyzing position and activity of a userin relation to the object. For example, a user touching the object,standing adjacent to the object, gesturing towards the object, and soforth may indicate the attention of the user is on that object. Thus theaugmented reality system may determine that the attention of the user isdirected to the barbell because he is standing next to and facing thebarbell.

At 1704, the identified object is associated with an advertisement. Insome implementations, this may involve a query of the advertisementdatastore 120 to determine what, if any, creatives 702 are associatedwith that object. Association may be based in some implementations on acomparison of the object's category 404(1) and the associated categories708 of the advertisement. Continuing our example, the barbell's and adfor “Ned's Virtual Gym” share the category of “exercise equipment.”

In some implementations, association of the object with an advertisementmay also be dependent upon identity of the user or a group of users. Forexample, adults may receive the ad for “Ned's” while children may see apublic service announcement for eating healthy snacks.

At 1706, the advertisement is presented within the augmented realityenvironment. This presentation may include audible, visual, or tactileoutputs by the augmented reality system. As shown here, the creative 702is presented on the wall adjacent to the barbell. The user interfacemodule 126 may be configured such that the presentation of theadvertisement is made relative to the positioning of the userinteracting with the object. For example, as shown here the creative 702is presented on the wall within the line of sight of the user, and notpresented on the television 1010 behind him. In some implementations theadvertisement may track with the user. For example, if the advertisementinvolves Cola, the advertisement may track so that it remains within theline of sight of the user while he is walking through rooms in theaugmented reality environment 1000.

The presentation of the advertisement may also be limited based at leastin part upon an identity or characteristics of one or more users withinthe augmented reality environment. For example, an adult female may notbe presented with advertisements for men's suits. Or certainadvertisements may be limited to presentation when children are present,such as for a toy.

The augmented reality system may be configured to cease presentation ofthe advertisement associated with the object when the user isinteracting with another object. For example, the user may turn awayfrom the barbell and get something out of the refrigerator. Upon turningaway, the ad for “Ned's” may fade or disappear. However, anotheradvertisement for something associated with the refrigerator may thenappear.

At 1708, a user input associated with the advertisement is accepted. Asabove, this input may comprise audible signals, gestures, or acombination thereof. For example, as shown here the user is using a handgesture indicating acceptance.

At 1710, an action is initiated based at least in part upon the userinput. These actions may include acknowledging the ad and requestingfurther action such as initiating a purchase of goods and services,accepting or declining the ad with various intensities, and so forth. Asshown here, the user's earlier acceptance action has prompted an actionwhereby the augmented reality environment has indicated acceptance ofthe advertisement and is providing additional information to the user on“Ned's” services with an option to sign up.

The particular gesture, or the magnitude thereof, may be used toindicate difference intensities of response. For example, a user may beslightly interested but not wish to view the ad at this particular time.A negative gesture with minimal motion may indicate that the intensityof response is low, and thus the ad is to be presented at a later time.A negative gesture made with large motions may indicate a high intensityof response and either increase the temporal delay in presenting theadvertisement again, or cause the ad to be removed from presentation tothis particular user permanently. These interaction details may bestored within the user interaction datastore 116, as described above.

CONCLUSION

Although the subject matter has been described in language specific tostructural features, it is to be understood that the subject matterdefined in the appended claims is not necessarily limited to thespecific features described. Rather, the specific features are disclosedas illustrative forms of implementing the claims.

What is claimed is:
 1. An augmented reality system comprising: aprocessor; a projector coupled to the processor and configured togenerate structured light within an environment; a camera coupled to theprocessor and configured to receive a portion of the structured lightthat has been reflected off one or more objects in the environment; atracking and control module coupled to the processor and configured to:identify, using the portion of the structured light, a user; determine,using the portion of the structured light, a gesture generated by theuser; identify an interaction with a physical object of the one or moreobjects in the environment based, at least in part, on a determinationthat the gesture is made towards the physical object; cause informationbased on the interaction to be output; identify, based at least in parton the information, a different physical object; associate the gesturewith the different physical object; and determine a price for thedifferent physical object; a user interface module coupled to theprocessor and configured to: generate, using the projector, anindication of the different physical object and the price, theindication being visually perceptible in the environment; and atransaction module coupled to the processor and configured to generatean offer for sale of the different physical object based at least inpart upon the gesture.
 2. The system of claim 1, wherein the physicalobject comprises an audio output device, the information comprises anadvertisement output by the audio output device, and the advertisementadvertises the different physical object.
 3. The system of claim 1,wherein the price comprises an offer price at which the differentphysical object is to be offered for acquisition.
 4. The system of claim1, wherein the gesture comprises one or more motions of the user thatoccur at least in part in free space free from contact with otherphysical objects within a pre-determined period of time.
 5. The systemof claim 1, wherein the transaction module is further configured toprovide the offer to a marketplace.
 6. The system of claim 1, whereinthe transaction module is further configured to store at least a portionof the offer in a transaction datastore.
 7. The system of claim 1,wherein the transaction module is further configured to: identify a userissuing the pre-determined gesture; and determine an authorization levelof the user; and wherein the offer is generated at least partly inresponse to determining that the authorization level of the user,authorizes the generating of the offer.
 8. One or more non-transitorycomputer-readable media storing computer-executable instructions that,when executed, cause one or more processors to perform acts comprising:causing augmented reality data to be captured from a camera;identifying, using the augmented reality data, a first user;identifying, using the augmented reality data, a pre-determined sellgesture, made by the user in an environment, comprising one or moremotions that occur at least in part in free space free from contact withobjects in the environment; and at least partly in response to theidentifying of the pre-determined sell gesture: identifying a subset ofthe objects in the environment, the subset comprising a first physicalobject and a second physical object; generating a first indicator and asecond indicator, the first indicator designating the first physicalobject in the environment and the second indicator designating thesecond physical object in the environment; identifying a selectiongesture, indicating a selection by the user of the first physical objectfrom among the first physical object and the second physical object;determining an offer price at which to offer the first physical objectfor acquisition at least in part by generating an appraisal of a valueassociated with the first physical object; indicating the offer price atwhich to offer the first physical object for acquisition; and generatingan offer for the first physical object.
 9. The one or morenon-transitory computer-readable storage media of claim 8, the offercomprising an offer to transfer property rights associated with thefirst physical object.
 10. The one or more non-transitorycomputer-readable storage media of claim 8, further comprising:accessing ownership rights for the physical object, wherein theownership rights include first ownership rights that indicate firstproperty rights for a first user and second ownership rights thatindicate second property rights for a second user; and determining,based at least on the first ownership rights, that the first user holdsproperty rights that allow the first physical object to be sold by thefirst user.
 11. The one or more non-transitory computer-readable storagemedia of claim 8, wherein the first indicator comprises a projectedimage proximate to the first physical object and the second indicatorcomprises a projected image proximate to the second physical object. 12.The one or more non-transitory computer-readable storage media of claim8, wherein one or more attributes of the first indicator are inproportion to a purchase price of the first physical object.
 13. The oneor more non-transitory computer-readable storage media of claim 8,wherein one or more attributes of the first indicator are in proportionto the offer price of the first physical object.
 14. The one or morenon-transitory computer-readable storage media of claim 13, wherein theone or more attributes comprise size, brightness, color, intensity, or acombination thereof.
 15. The one or more non-transitorycomputer-readable storage media of claim 8, further comprising queuingoffers from one or more unauthorized users for confirmation by anauthorized user.
 16. The one or more non-transitory computer-readablestorage media of claim 8, further comprising providing the offer to amarketplace.
 17. The one or more non-transitory computer-readablestorage media of claim 8, further comprising modifying the firstindicator designating the first physical object when local availabilityof the object exceeds a pre-determined threshold.
 18. The one or morenon-transitory computer-readable storage media of claim 8, furthercomprising ordering shipping materials.
 19. The one or morenon-transitory computer-readable storage media of claim 10, wherein thefirst ownership rights differ from the second ownership rights.
 20. Theone or more non-transitory computer-readable storage media of claim 8,further comprising determining a location of the first physical object,and storing the location of the first physical object.
 21. One or morenon-transitory computer-readable media storing computer-executableinstructions that, when executed, cause one or more processors toperform acts comprising: causing augmented reality data to be capturedfrom a camera; identifying, based at least in part on the augmentedreality data, a pre-determined sell gesture made by associated with auser in an environment; at least partly based on identifying thepre-determined sell gesture, identifying a first object in theenvironment and a second object in the environment; generating a firstindicator and a second indicator, the first indicator designating thefirst object in the environment and the second indicator designating thesecond object in the environment; identifying a selection gestureindicating a selection by the user of the first object from among thefirst object and the second object; associating the pre-determined sellgesture with the first object in a physical environment, wherein thepre-determined sell gesture comprises one or more motions that occur atleast in part free from contact with other objects; at least partly inresponse to the associating: accessing ownership rights associated withthe first object; determining, based at least in part on the ownershiprights, that the user is authorized to sell the first object;determining an offer price at which to offer the object for acquisitionat least in part by generating an estimate or projection of a valueassociated with the first object; using a projector disposed in theenvironment, presenting a visible indication at a location of the firstobject in the environment, the visible indication being projected ontoor proximate the first object and indicating the offer price at which tooffer the first object for acquisition; and generating an offer fortransfer of one or more property interests associated with the firstobject.
 22. The one or more non-transitory computer-readable storagemedia of claim 21, the property interests comprising possession of thefirst object, ownership of the first object, a lien associated with thefirst object, or a combination thereof.
 23. The one or morenon-transitory computer-readable storage media of claim 21, furthercomprising: presenting the offer to one or more possible purchasers; andreceiving a response to the offer from the one or more possiblepurchasers.
 24. The one or more non-transitory computer-readable storagemedia of claim 21, wherein the ownership rights include first ownershiprights that indicate first property rights for the user and secondownership rights that indicate second property rights for another user;and wherein determining that the user to authorized to sell the firstobject comprises identifying that the first ownership rights indicatethat the user holds property rights that allow the first physical objectto be sold by the user.
 25. The one or more non-transitorycomputer-readable storage media of claim 21, further comprisingreceiving the pre-determined sell gesture in an augmented realityenvironment.
 26. The one or more non-transitory computer-readablestorage media of claim 21, the first object comprising a virtual object.27. The one or more non-transitory computer-readable storage media ofclaim 21, the first object comprising a physical object.
 28. The one ormore non-transitory computer-readable storage media of claim 21, whereinthe first indicator designating comprises presenting a visual indicatorproximate to the first object and the second indicator comprises avisual indicator proximate to the second object.
 29. One or morenon-transitory computer-readable media storing computer-executableinstructions that, when executed, cause one or more processors toperform acts comprising: causing one or more of a camera or a microphoneto capture augmented reality data from an environment; identifying,based at least in part on the augmented reality data, a pre-determinedsell gesture within the environment, wherein the sell gesture indicatesto initiate a transfer of a property interest associated with an object;at least partly in response to the pre-determined sell gesture and theaugmented reality data, identifying a plurality of objects within theenvironment, the plurality of objects comprising a first object and asecond object; generating a first indicator designating the first objectin the environment and a second indicator designating the second object,the first indicator being projected into the environment onto orproximate the first object and the second indicator being projected intothe environment onto or proximate the second object; identifying aselection gesture, indicating a selection by the user of the firstobject from among the first object and the second object; accessingownership rights associated with the first object, wherein the ownershiprights include first ownership rights that indicate first propertyrights for a first user and second ownership rights that indicate secondproperty rights for a second user; determining, based at least on thefirst ownership rights or the second ownership rights, that a userassociated with the pre-determined sell gesture holds property rightsthat allow the first object to be sold; and generating an appraisal of avalue associated with a transfer of the property interest associatedwith the first object, the appraisal being an estimate or projection ofthe value associated with the transfer of the property interestassociated with the first object.
 30. The one or more non-transitorycomputer-readable storage media of claim 29, further comprising at leastpartly in response to receiving an acceptance gesture from the user,offering a transfer of the property interest associated with the firstobject.
 31. The one or more non-transitory computer-readable storagemedia of claim 29, wherein the generating the appraisal comprisesquerying a transaction datastore.